An Entropy Maximization Approach to Optimal Model Selection in Gaussian Mixtures

In this paper we address the problem of estimating the parameters of a Gaussian mixture model. Although the EM (ExpectationMaximization) algorithm yields the maximum-likelihood solution it has many problems: (i) it requires a careful initialization of the parameters; (ii) the optimal number of kernels in the mixture may be unknown beforehand. We propose a criterion based on the entropy of the pdf (probability density function) associated to each kernel to measure the quality of a given mixture model, and a modification of the classical EM algorithm to find the optimal number of kernels in the mixture. We test this method with synthetic and real data and compare the results with those obtained with the classical EM with a fixed number of kernels.